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Abstract. The symmetric group on 4 letters has the reflection group D3 as an isomor- 
phic image. This fact follows from the coincidence of the root systems A3 and D%. The 
isomorphism is used to construct an orthogonal basis of polynomials of 4 variables with 2 
parameters. There is an associated quantum Calogero-Sutherland model of 4 identical par- 
ticles on the line. 
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1 Introduction 

The symmetric group on N letters acts naturally on M. N (for N = 2,3, . . .) but not irreducibly, 
because the vector (1,1,..., 1) is fixed. However the important basis consisting of nonsymmetric 
Jack polynomials is defined for N variables and does not behave well under restriction to the 
orthogonal complement of (1, 1, ... , 1), in general. In this paper we consider the one exception 
to this situation, occurring when N = 4. In this case there is a coordinate system, essentially 
the 4x4 Hadamard matrix, which allows a different basis of polynomials, derived from the 
type-.B nonsymmetric Jack polynomials for the subgroup D3 of the octahedral group B3. We 
will construct an orthogonal basis for the L 2 -space of the measure 

Y\ \xi - Xj\ 2K \x\ + x 2 + x 3 + x 4 | 2k exp I ^2 x i ) ^ x 

on M 4 , with k,k' > 0. 

We will use the following notations: No denotes the set of nonnegative integers; Nq is the set 
of compositions (or multi-indices), if a = (a±, . . . , «at) G Nq then |a| := X^=i Q « an d the length 
of a is £(a) := max{i : on > 0}. Let Nq ,+ denote the subset of partitions, that is, A € Nq and 
\ > Aj+i for 1 < i < N. For a E Nq and x £ M. N let x a = YiiLi X T '' ■> a monomial of degree |a|; 
the space of polynomials is V = span K {x a : a G Nq^}. For x,y G W N the inner product is 
{x,y) := Y^i=i x iyii an d \x\ := {x,x) 1 ^ 2 ; also x 1 - := {y : (x,y) = 0}. The cardinality of a set E 
is denoted by f^E. 

Consider the elements of Sjy as permutations on {1,2,... ,N}. For x G and w G Sn 
let (xw) i := for 1 < i < N and extend this action to polynomials by (wf) (x) = f (xw). 
Monomials transform to monomials: w (x a ) := x wa where (to); := a UI -i(j) for a G Nq . (Con- 
sider x as a row vector, a as a column vector, and w as a permutation matrix, with l's at the 



*This paper is a contribution to the Special Issue on Dunkl Operators and Related Topics. The full collection 
is available at |http: / / www.emis.de /journals/SIGMA /Dunkl_operators.html| 



2 



C.F. Dunkl 



(w (j) ,j) entries.) For 1 < i < N and / G V the Dunkl operators are 

WW ■■=^-f{ X ) + ^ nx) ~ f{x{h3)) ^ 



3T* 



and 



t-i 

Uif (x) := T>i ( Xi f {x))-kJ2 (j, i) f 0) • 

3=1 

Then UilAj = Ujlii for 1 < i,j < N and these operators are self-adjoint for the following 
pairing 

{f,9) K /(^l! • • • ,V N )g(x) | x=0 . 

This satisfies (/,c/) K = (ff,/)* = (wf,wg) K for f,g eV and w G S^; furthermore (/,/) K > 
when / and k > 0. The operators Hi have the very useful property of acting as triangular 
matrices on the monomial basis furnished with a certain partial order. However the good 
properties depend completely on the use of 1^ even though the group Sn acts irreducibly on 
(1,1,...,1). We suggest that an underlying necessity for the existence of an analog of {Ui\ 
for any reflection group W is the existence of a W-orbit in which any two points are orthogonal 
or antipodal (as in the analysis of the hyperoctahedral group Bn). This generally does not 
hold for the action of Sn on (1, . . . , 1) . We consider the exceptional case N = 4 and exploit 
the isomorphism between 1S4 and the group of type D 3 , that is, the subgroup of B 3 whose 
simple roots are (1,-1,0), (0,1,-1), (0,1,1). We map these root vectors to the simple roots 
(0,1,-1,0), (0,0,1,-1), (1,-1,0,0) of ^4, in the same order. This leads to the linear isometry 

2/1 = 5 (sci + x 2 - x 3 - x 4 ) , 
V2 = \ (xi -x 2 + x 3 - X A ) , 
2/3 = 5 {xi -x 2 -x 3 + x A ) , 

2/0=2 ( x l +X 2 + X 3 + Xi) . (1) 

Consider the group D 3 acting on (2/1,2/212/3) and use the type- S3 Dunkl operators with the 
parameter k! = (associated with the class of sign-changes yi 1— > —yi which are not in D 3 ). 
Let (Tij, Tij denote the reflections in yi — yj — 0, yi + yj — respectively. Then for i — 1, 2, 3 let 

dyi . . V v%- Vj Vi + yj J 

Uff (y) = Vf (yj (y)) - « £ (^ + Tij ) f (y) . 

i<j<i 

The operators j^ 8 } commute pairwise and are self-adjoint for the usual inner product. The 
simultaneous eigenvectors are expressed in terms of nonsymmetric Jack polynomials with ar- 
gument (2/f, 2/21 2/|) • I n the sequel we consider polynomials with arguments x or y with the 
convention that y is given in terms of x by equation ([I]). 

2 Nonsymmetric Jack polynomials 

Nonsymmetric Jack polynomials (NSJP) are the simultaneous eigenfunctions of {Ui}^ 1 . We 
consider the formulae for arbitrary N since there is really no simplification for N = 3. 
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Definition 1. For a G Nq , let a + denote the unique partition such that a + = wa for some 
w G Sj\r- For a, f3 G Nq the partial order a >- [3 (a dominates j3) means that a f3 and 
Yli=i a i — Yli=i Pi f° r 1 — J ' — N; a > means that \a\ = |/3| and either a + >- f3 + or a + = f3 + 
and a >- /3. 

For example (2, 6, 4) o (5, 4, 3) > (3, 4, 5). When acting on the monomial basis {i° : a £ N^, 
|a| = n} for n G No the operators have on-diagonal coefficients given by the following 
functions on N V . 

Definition 2. For a G and 1 < i < N let 

r (a, i) := # {j : ay > + # {j : 1 < j < i, ay = , 
(a) := (N - r (a, i)) k + a./ + 1. 

Clearly for a fixed a G the values {r (a, i) : 1 < i < iV} consist of all of {1, ... , N}; let «; 
be the inverse function of % \— ► r (a, i) so that w G Sn, r (a, w (i)) = i and a + = wa (note that 
a G if and only if r (a, i) = i for all z). Then 

UiX a = & (a) x a + (x) 

where q a ^ (x) is a sum of terms ±kx@ with a [> /?. 

Definition 3. For a G N^, let £a denote the x-monic simultaneous eigenfunction (NSJP), that 
is, UiQ a = (a) ( a for 1 < i < N and 

Ca = x + ^ ] Ap a X^ , 

with coefficients ^4/3^ G Q(k), rational functions of k. 

There are norm formulae for the pairing (•, •) . Suppose a G and £ (a) = m; the Ferrers 
diagram of a is the set {(i, j) :l<i<m,0<j< a,;} . For each node (i, j) with 1 < j < ai 
there are two special subsets of the Ferrers diagram, the arm : j <l < a,} and the leg 

■ I > i, j < oli < ai} U — I) : I < i, j — I < ai < ai}. The node itself, the arm and 
the leg make up the hook. (For the case of partitions the nodes (i, 0) are customarily omitted 
from the Ferrers diagram.) The cardinality of the leg is called the leg- length, formalized by the 
following: 

Definition 4. For a G N^, 1 < i < t (a) and 1 < j < ai the leg- length is 

L(a;i,j) : = # {/ : I > i, j < a t < + # {I : I < i, j < a t + 1 < . 
For t G Q (k) the hook-length and the hook-length product for a are given by 
h(a,t;i,j) := (ai - j + t + kL (a\i, j)) , 

h(a,t) := J ~[h(a,t;i,j) , 

i=l j=l 

and for A G and f G Q (k) the generalized Pochhammer symbol is 

N Xi-1 

i=l j=0 
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(The product over j is an ordinary Pochhammer symbol.) 
Proposition 1. For a,f5 G N^, the following orthogonality and norm formula holds: 

Details can be found in the book by Xu and the author [2i Chapter 8], the concept of leg- 
length and its use in the norm formula is due to Knop and Sahi [3]. The (symmetric) Jack 
polynomial with leading term x A for A € N^' + is obtained by symmetrizing (*\. The coefficients 
involve, for a E N^, e = ±1: 

en 



8 £ {a):= J] (l + 

i<j,cti<ctj 

in fact, [U Lemma 3.10], 



(r (a, i) — r (a,j)) k + ay — / 



/i (a, At + 1) = £i (a) /i (a + , At + l) , 
/i (q + , l) = h (a, 1) 6-i (a) , 



for aeNf. Then 



h = ^2 £ -i («) Coo 

(iY K + l) A / i (A,l) 



(j\,j\) K = #{a:a + = X] 



E x (\ R ) h (A, k + 1) ' 

where A^ = Atv+i-j for 1 < i < N (the reverse of A). Note {a : a + = A} = {w\ : w £ 5/v}. 



3 The groups S4 and D 3 

By using the x <-> y correspondence (equation ([I])) we obtain operators which behave well on 
(1, . . . , l)" 1 ". Here are the lists of reflections in corresponding order: 

[fl2,Tl2, 013) T~l3, 0~23, T23} ; 

[(23), (14), (24), (13), (34), (12)]. 

The following orthonormal basis is used in the directional derivatives: 

v = l(l, 1,1,1), 
ui = 1,-1,-1), 

U 2 = i (1,-1, 1,-1), 
v 3 = l (1,-1, -1,1). 

That is, yj = (a;, Vi) and ^- = X)j=i ( v i)j ^f- for < z < 3. Note that {±v 1 ,±v 2 ,±v 3 } is an 
octahedron and an 54-orbit. Then 

/ — ' J dx 7 - V x 2 - £3 xi — 24 j; 2 — X4 X1 — X3 J 

7=1 J 
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and similar expressions hold for T>2 , ■ Furthermore 
U?f{x) = V?{(v x ,x)f{x)), 

Uff (x) = V* ((v 2 ,x) f (X)) - k ((14) + (23)) / (x) , 

Uif (x) = Vi ((v 3 , x) f (x)) - k ((12) + (13) + (24) + (34)) / (x) . 

For a subset E C {1, 2, 3} let ue = IlieE Vii a ^ s0 E = and E k = {1, . . . ,k} for k = 1, 2, 3. 
The simultaneous eigenfunctions are of the form VeI (y 2 ) where y 2 := (2/1 5 2/i' an d when 
E = Ek with < k < 3 they are directly expressed as NSJP's (for M 3 ). The following is the 
specialization to k' = of the type-i? result from [21 Corollary 9.3.3, p. 342]. 

Proposition 2. Suppose a G Nq and A; = 0, 1, 2, 3, i/ien /or 1 < i < k 

K B VE k Ca (y 2 ) = 26 (a) 2/ Bfc Ca (y 2 ) , 

and /or k < i < 3 

U?y Ek Ca {y 2 ) = (26 (a) - 1) y Ek Ca {y 2 ) ■ 

The polynomial yE k Ca {y 2 ) is labeled by (3 G Nq where = 2«j + l for 1 < i < and = 2a, L 
for k < i < 3. The difference /3 — a G Nq and appears in the norm formula (the result for the 
pairing (/,<?) 1— > / (D^,P^,D^) 5 (y) | y=0 applies because of the isomorphism). 

Proposition 3. Suppose (3 G Nq and is odd /or 1 < i < k and is even otherwise, then for 



a, 



'o 

Pi ' 
2 



, 1 <i < 3 



(The formulae in [21 Chapter 9] are given for the p-monic polynomials, here we use the x- 
monic type, see [21 pp. 323-324]). There is an evaluation formula for £ Q (1, 1, 1) which provides 
the value at x = (2, 0, 0, 0) , corresponding to y = (1, 1, 1, 1). Indeed for a G Nq (see [21 p. 324]) 

um,d- (3k + 1u 



/i (a, k + 1) 



For any point (±2, 0, 0, 0) w with w £ S4 the corresponding y satisfies yi = ±1 for 1 < i < 3, 
so that y 2 = (1, 1, 1). For any other subset E C {1, 2, 3} with = k let m G £3 be such that 
w (i) £ E for l<i<fc, l<i<j</eorfc<i<j<3 implies u; (i) < w (j) (that is, w 
preserves order on {1, . . . , k} and on {k + 1, . . . , 3}). Here is the list of sets with corresponding 
permutations (to(i))? =1 : 



E = 


{2}, 


w 


= (2,1,3) 


E = 


{3}, 


w 


= (3,1,2) 


E = 


{1,3}, 


w 


= (1,3,2) 


E = 


{2,3}, 


w 


= (2,3,1) 



Then (letting w act on y) wyE k = yE and for a G Nq the polynomial w (yE k (a {y 2 )) is a simul- 
taneous eigenfunction and 

U w{i) w yE k Ca (y 2 ) = 26 (a) wy Ek Q a (y 2 ) , l<i<k, 
U%(i)™VE k (a (y 2 ) = (26 (a) - 1) wy Ek ( a (y 2 ) , ft < % < 3. 
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Define (5 as before (/% = 2aj + 1 for 1 < i < k and Pi = 2a{ for k < i < 3) then the label for the 
polynomial wyE k Ca {y 2 ) is w(3 (recall (w/3)j = (3 w ~i^)- Denote 

Pw/3 (y) := wy Ek ( a (y 2 ) . 

This defines a polynomial p 7 for any 7 6 Nq. The norm of wyE k Ca (y 2 ) is the same as that 
of yE k Ca {y 2 ) since any w G £3 acts as an isometry for (•, -) K . Suppose -E' C {1,2,3} and 
E ^ E' and f,g£ then (?/_e/ (y 2 ) , 2/-E'5 (?/ 2 )) = 0- The root system D3 is an orbit of the 
subgroup of diagonal elements of B% (isomorphic to Denote the sign change yi 1 — > — 

by (Tj for 1 < z < 3. From the -B3 results we have <7jP^ = ^f a i f° r 1 < < 3 and this 

implies (y E f (y 2 ) ,ve'9 {y 2 )) K = (viVEf {y 2 ) ,cnyE>9 (y 2 )) K = - {ve! {y 2 ) ,VE>g (y 2 )) K for an y 

i G (E\E') U (E'\E) (the symmetric difference). Thus {p 7 : 7 G Njj} is an orthogonal basis 
for <•,•>„■ 

We consider the 54-invariant polynomials: they are generated by yo, J2i=i 2/ 2 , 2/12/22/3, 2~2i=i Vi- 
Any invariant is a sum of terms of the form yfi (yiy22/3) S / {y 2 ) where n G No, s = or 1, and / 
is a symmetric polynomial in three variables. For now consider only polynomials in {2/1,2/2,2/3}- 
Let A G Nq + , then there are two corresponding simultaneous eigenfunctions of Yli=i (^ B )" 
suffices to take n = 1, 2, 3 to generate the commutative algebra of 54-invariant operators). From 
[21 Theorem 8.5.10] let 

^ = #{a:a+ = A} (*■+!)* *(*.D 



fi (A fi ) /» (A, k + 1) ' v ' 

f°x (*) = h (y 2 ) , 

F x ( x ) = ymy-sjx {y 2 ) , 

<^ 1 ^A 1 > K = 2 2 ^(2. + i) A X . 

V ^/ (Ai+1,A 2 +1,A 3 +1) 

The polynomials : A G Nq' + | are pairwise orthogonal. 

Up to now we have mostly ignored the fourth dimension, namely, the coordinate yo- The 
reflection gq along vo (given by xgq = x — f^iLl x i) v o) commutes with the 54-action. We 
introduce another parameter k 1 and let 

v °f (*) = 5 E #-/ (*) + "r~~~~T (/ (*) " / , 



P</ (x) = VJ (x) + (/ (x) - f (xa )) • 

2 (x,u ) 

The operators \D' i : 1 < i < 4} are the Dunkl operators for the group W = S4 x Z2 (the 
reflection group generated by {(1, 2) , (2, 3) , (3, 4) , <r })- Then £> 2/o n = 2nj/ 2n_1 and V yl n+l = 
(2n + 1 + 2k') yl n . We define the extended pairing for polynomials 

(f(x),g(x)) K:Kl = f{V[,...,V',)g(x)\ x=0] 

in terms of y 

ifo (yo) fi (2/1,2/2, 2/3), 50 (2/0)51 (2/1, 2/2, 2/3 ))«,«/ 

= /o (£>o)270 (2/0) |y =0 X /l (£>f , •••) 51 (2/1,2/2,2/3) |y=0 
= /o (^0)50 (2/o) |y =0 X (/l,5l)« • 
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It is easily shown by induction that for n G No 

V / n+. 



The direct product structure implies that {p( 7l ,72,73) (.y)Uo 4 : 7 S Nq} is an orthogonal basis 



4 Hermite polynomials 

The pairing (•,•)««' 1S related to a measure on K 4 : let k, k' > and 
dm (a;) := (2vr)~ 2 exp (-- \x\ 2 jdx, x E K 4 , 



/i(x) := |xj - xj\ K \y Q \ K , 

l<i<j<4 



d^ K ,K' (x) := c K , K //i (z) 2 dm (x) . 
In fact 

_! _ , r («' + 1) r (2k + 1) r (3k + 1) r (4k + 1) 
~ r(i)r( K + i) 3 ' 

The integral is a special case of the general formula (any suitably integrable function / on 

N 



(2tt) N/2 [ TT \xi - Xj\ 2K f ( y^Xi ) exp ( -- \x\ 2 )dx 

jRN i<ikf<N \t! J V 2 / 

y/toJ-ao \ > fi r (K + 1) 



this follows from the Macdonald-Mehta-Selberg integral for Sn and the use of an orthogonal 
coordinate system for R. N in which Ei=i x * I '>/~N is one of the coordinates. The Laplacian is 

A. := Etl = ELi (^f) 2 + ^o- Also set A B := £* = i (Z>?) 2 - Then for /, <? £ V U 

Theorem 5.2.7] 



(f,9) K , K > = f (e~ Ah/2 f(x)) (e- A ^ 2 g(x)) d» K>K , (x) . 



The orthogonal basis elements p y (y) (7 G Njj,n G No) are transformed to orthogonal poly- 
nomials in L 2 (R 4 ,/i K K /) under the action of e~ Ah ^ 2 = Yl^Lo h H)"^ (only finitely many 
terms are nonzero when acting on a polynomial). We have 
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Then for n G N 



Recall the Laguerre polynomials {L^ (t) : n G No} are the orthogonal polynomials for the mea- 
sure t a e~*(it on {t : t > 0} with a > —1, and 



The result of applying e -Afl / 2 to a polynomial XE k (a {y 2 ) is a complicated expression involving 
some generalized binomial coefficients (see [21 Proposition 9.4.5]). For the symmetric cases 
j\ (?/ 2 ) an d y\V2Vz3\ {y 2 ) > A G Nq' + these coefficients were investigated by Lassalle [1] and 
Okounkov and Olshanski [5j equation (3.2)]; in the latter paper there is an explicit formula. 

Finally we can use our orthogonal basis to analyze a modification of the type- A quantum 
Calogero-Sutherland model with four particles on a line and harmonic confinement. By resca- 
ling, the Hamiltonian (with exchange terms) can be written as: 



x\ 2 sr^ 4k/ (k' - cr ) 



x v — \ n, — it,/) 



l<i<j<4 ( X i ~ x jY ( x l + x 2 + X 3 + X 4 ) 2 

When this is applied to a ^-invariant the reflections and Co are replaced by the scalar 1. 
We combine the type-B results from [21 Section 9.6.5] (setting k' = in the formulae) with 
simple Z2 calculations. The nonnormalized base state is 



l<i<j<4 

Then 

3 

"1 



^-^^ = _a b - ^0 + E W^T + 6k + + 2. 



8=0 

This operator has polynomial eigenfunctions and the eigenvalues are the energy levels of the 
associated states. From [21 Section 9.6.5] we have 

e -A B/ 2^2 u B e A B/ 2 = _ Ab + £ B +( , k + ^ 

and by direct calculations 

V 2_ d 2 | 2k' d j I-oq 
° dyl y dy y\ 

e^' 2 (V y - K'a Q ) e^ 2 = -V\ + y JL +n' + l. 

dyo 

Combine these results: 

= e-^' 2 ^>f + v °y° - K ' a ° - 2 ) eAh/2 - 
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Thus (e A,l//2 (p 7 (y) y^)) ?/>o is an eigenfunction of H for each 7 G Nq, n G No- It suffices to 
consider yE k (a (y 2 ) y$- We have 

(P yo - «V ) yg n = ((2n + 1 + 2k') - «') y 2 ", 
{V y - «V ) y 2n+1 = ((2^ + 2) + «') y 2 ", 
(P02/0 - kV ) y£ = (n + 1 + «') 

Furthermore Ei=i W f (l/B*Ca (?/ 2 )) = (2EL1 6 H ~ ( 3 ~ J/£ fe Ca (y 2 ); the eigenvalue is 
(2 |a| + fe) + 6k + 3 = + 6k + 3 (where ft = 2a 4 + 1 for 1 < « < fc and A = 2^ for 
< i < 3). The energy level for (e~ Ah / 2 (p^ (y) y^)) V'o is |/3| + n + 6k + k' + 2. Observe the 
degeneracy of the energy levels; only the total degree \j3\ + n appears. The (nonnormalized) 
VF-invariant eigenfunctions are (A G Njj) 

(e- A ^(j x (y 2 ))L£-V* (f) )*>(*), 

(e" A */ 2 (yiy 2 y 3 jA (y 2 )) L^ 2 (f )) V>o (*) . 

The L 2 -norms can be found by using equation ([2]). 

In conclusion, we have found an unusual basis for polynomials which allowed an extra pa- 
rameter in the action of £4 on M 4 . This exploited the fact that Vq has an orthogonal basis 
which together with its antipodes forms an 54-orbit. The pairing (•,■)„ has an analog for each 
reflection group and weight function. We are left with the interesting problem of how to con- 
struct orthogonal bases for groups not of type A or B. 
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